Safety-critical computer vision: an empirical survey of adversarial evasion attacks and defenses on computer vision systems

Abstract Considering the growing prominence of production-level AI and threat adversarial attacks that can poison a machine learning model against certain label, evade classification, or reveal sensitive data about training to an attacker, adversaries pose fundamental problems systems. Furthermore, much research has focused on inverse relationship between robustness accuracy, raising for real-time safety-critical systems particularly since they are governed by legal constraints in which software changes must be explainable every change thoroughly tested. While many defenses have been proposed, often computationally expensive tend reduce accuracy. We therefore conducted large survey present simple practical framework analyzing any machine-learning system from perspective using noise find upper bound failure rate. Using this method, we conclude all tested configurations ResNet architecture fail meet reasonable definition ‘safety-critical’ when even small-scale benchmark data. examine state art computer vision with focus applications autonomous driving, industrial control, healthcare. By testing combination defenses, their efficacy, run-time requirements, provide substantial empirical evidence modern neural networks consistently established standards wide margin.